[GuideLLM Refactor] scheduler package updates, rewrites, and tests expansion #354
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
markurtz
requested review from
DaltheCow,
sjmonson,
markVaykhansky,
AlonKellner-RedHat and
Copilot
There was a problem hiding this comment.
Pull Request Overview
This PR introduces a comprehensive refactor of the scheduler package to support distributed benchmarking with a flexible constraint system and enhanced timing control. The implementation moves from hardcoded execution limits to a composable constraint architecture and changes request timing calculations from precalculated to per-request basis for dynamic rate adjustments.
Key changes:
- Added a flexible constraints system with Protocol-based architecture supporting request limits, duration limits, and error thresholds
- Refactored core scheduler objects to be fully generic and decoupled from backend-specific types
- Enhanced scheduling strategies with per-worker timing instances instead of precalculated schedules
Reviewed Changes
Copilot reviewed 17 out of 18 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/scheduler/test_worker_group.py | Comprehensive test coverage for WorkerProcessGroup multi-process orchestration |
| tests/unit/scheduler/test_worker.py | Full test suite for WorkerProcess individual worker functionality |
| tests/unit/scheduler/test_strategy.py | Extensive testing of scheduling strategies and timing implementations |
| tests/unit/scheduler/test_scheduler.py | Core scheduler singleton testing with distributed coordination |
| tests/unit/scheduler/test_objects.py | Complete test coverage for scheduler objects and type relationships |
| tests/unit/scheduler/test_environment.py | Testing of environment abstractions for distributed coordination |
| tests/unit/scheduler/test_constraints.py | Comprehensive constraint system testing with factory patterns |
| src/guidellm/scheduler/worker_group.py | Multi-process worker orchestration implementation |
| src/guidellm/scheduler/worker.py | Individual worker process management implementation |
| src/guidellm/scheduler/strategy.py | Request scheduling strategies with timing abstractions |
| src/guidellm/scheduler/scheduler.py | Thread-safe singleton scheduler for distributed coordination |
| src/guidellm/scheduler/result.py | File removed as functionality moved to objects.py |
| src/guidellm/scheduler/queues.py | File removed as functionality integrated into worker components |
Comments suppressed due to low confidence (2)
tests/unit/scheduler/test_objects.py:1
- This test expects TypeError but ScheduledRequestInfo has Field defaults defined for all required fields. The test should either provide invalid field types or check for ValidationError instead.
from __future__ import annotations
tests/unit/scheduler/test_objects.py:1
- The field name
created_atdoesn't exist in ScheduledRequestInfo. This should bescheduler_start_timebased on the model definition.
from __future__ import annotations
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
markurtz
force-pushed
the
features/refactor/scheduler
branch
from
8597abc to
2d94201
Compare
Signed-off-by: Mark Kurtz <[email protected]>
…view Signed-off-by: Mark Kurtz <[email protected]>
markurtz
force-pushed
the
features/refactor/scheduler
branch
from
2d94201 to
a7ae737
Compare
sjmonson
approved these changes
Sep 23, 2025
sjmonson
added a commit
that referenced
this pull request
Sep 23, 2025
…o features/refactor/base-draft [GuideLLM Refactor] scheduler package updates, rewrites, and tests expansion #354
sjmonson
added a commit
that referenced
this pull request
Sep 25, 2025
…pansion \#354\n\nfeatures/refactor/scheduler
Base automatically changed from
features/refactor/utils
to
features/refactor/base
September 29, 2025 14:18
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Introduces a comprehensive constraints system and enhanced timing control for the scheduler refactor. The implementation moves from hardcoded execution limits to a flexible, composable constraint system that enables sophisticated benchmark stopping criteria. Additionally, request timing calculations are moved from precalculated to per-request basis, enabling dynamic rate adjustments and better distributed coordination.
Details
constraints.py): Implements Protocol-based constraint architecture with support for request limits, duration limits, error thresholds, and sliding window error ratesMaxNumberConstraint: Limits execution based on request countMaxDurationConstraint: Limits execution based on time durationMaxErrorsConstraint: Limits execution based on absolute error countMaxErrorRateConstraint: Limits execution based on sliding window error rateMaxGlobalErrorRateConstraint: Limits execution based on global error rateConstraintsInitializerFactory: Registry system for constraint creation and serializationobjects.py): Replacedresult.pyand expanded capabilitiesBackendInterfaceprotocol for type-safe backend integrationScheduledRequestInfowith comprehensive timing and status trackingSchedulerStatefor distributed state coordinationSchedulerUpdateActionfor constraint-based control signalsstrategy.py): Introduced request timing abstractionsScheduledRequestTimingsbase class for timing implementationsLastCompletionRequestTimings: For synchronous and concurrent strategiesNoDelayRequestTimings: For maximum throughput strategiesConstantRateRequestTimings: For fixed-rate schedulingPoissonRateRequestTimings: For stochastic request patternsenvironment.py): Coordination layer for distributed executionEnvironmentprotocol for distributed synchronizationNonDistributedEnvironmentimplementation for single-node executionworker.py,worker_group.py): Distributed request processing infrastructureTest Plan
Related Issues
Use of AI
## WRITTEN BY AI ##)